How to Evolve Safe Control Strategies 
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Abstract 

Autonomous space vehicles need adaptive control strate- 
gies that can accommodate unanticipated environmental 
conditions. The evaluation of new strategies can often be 
done only by actually trying them out in the real physical 
environment. Consequently, a candidate control strategy 
must be deemed safe — i.e., it won't damage any systems — 
prior to being tested online. How to do this efficiently has 
been a challenging problem. 

We propose using evolutionary programming in con- 
junction with a formal verification technique (called model 
checking) to evolve candidate control strategies that are 
guaranteed to be safe for implementation and evaluation. 



1. Introduction 

Control strategies are critical ingredients of a space mis- 
sion because they indicate what actions are to be taken by 
the spacecraft in response to environmental conditions. Un- 
fortunately, control strategies defined at the beginning of a 
mission may have to be modified later on. The need for this 
modification may be due to system failures that reduce func- 
tionality or because the spacecraft has encountered unantic- 
ipated environmental conditions. 

An appealing method for dealing with these undesirable 
situations is to use a reconfigurable system, which can adopt 
a different functionality. For example, a reconfigurable sys- 
tem eliminates the need for redundant hardware — which 
consumes precious space and weight — by simply modify- 
ing the existing hardware to compensate for the failure. 
However, despite the enormous advantages of reconfigu- 
ration, reconfiguration information originating from Earth 
will probably not arrive in time to do any good. 

The real solution lies with adaptive systems — i.e., sys- 
tems capable of self-reconfiguration in response to faults or 
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a changing operational environment |JTp. This adaption is 
performed in-situ (in place), thereby removing any reliance 
on Earth-bound resources for reconfiguration information. 

In this paper we propose a method for evolving new con- 
trol strategies in a way guaranteed to be safe during the re- 
configuration process. Our method fully supports in-situ 
adaption of the strategies. 

2. Discussion 

Control strategies can be evolved extrinsically, where 
each strategy is simulated, but only the best one is actually 
implemented, or intrinsically, where each candidate strat- 
egy is downloaded into the system and exercised in the real 
physical environment. In-situ extrinsic evolution may be 
problematic because some closed-form objective function 
is necessary to assess efficacy, but it may not always be 
possible to define a suitable one. Thus, in most cases in- 
trinsic evolution may be the only thing that makes sense. 
It is therefore absolutely essential that the control strategy 
be safe — i.e., it does no harm to the controller itself nor to 
any other system. This safety check must be made prior to 
testing the new strategy online. 

Our approach is to evolve a series of deterministic finite 
state machines (FSMs), each encoding a potential new con- 
trol strategy. Evolutionary programming (EP) [|]| is used to 
evolve these FSMs. The suitability of each strategy will be 
assessed by actually trying it in the real physical environ- 
ment. However, only control strategies that pass a safety 
check will be downloaded for evaluation. We will borrow 
automatic formal verification methods to assess this safety. 
These methods use mathematically provable techniques to 
characterize a system without conducting exhaustive simu- 
lation or testing. Specifically, we will rely on model check- 
ing (MC) techniques [|]| to verify the safety of candidate 
FSMs generated by EP. Although model checking has been 
extensively used in hardware design and software verifica- 
tion, to the best of our knowledge no prior research effort 
in formal methods has attempted the problem we consider 



here. 

MC is a formal method that verifies if a system, mod- 
elled as a FSM, adheres to a specified property. The prop- 
erties of interest are encoded as temporal logic expressions, 
which expresses properties that change over time [Q]. There 
are many different kinds of temporal logic but computation 
tree logic (CTL) is the most widely used with model check- 
ers. The basic idea is a safety property is expressed in ordi- 
nary Boolean logic, and then special temporal operators are 
added for describing future events. 

MC has been used to verify properties in systems with 
hundreds of thousands of states. In practice, control strate- 
gies tend to have orders of magnitude fewer states. The MC 
algorithm complexity is linear in the size of the FSM and 
in the length of the CTL expression [Q], so the safety of a 
control strategy can be quickly verified. A graphical repre- 
sentation of the MC algorithm is shown in Figure [j]. 
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Figure 1. A graphical depiction of the model 
checking algorithm. The control strategy is 
described by a FSM. I is the set of all FSM 
initial states and Y is the set of FSM states 
that violates a safety property. The algorithm 
recursively computes Y i+ i = Pre(Fi) U Y { for 
i = 0, 1, 2, . . . n - 1 where Pre(Fi) is the preim- 
age of the set Y { . Y n then represents the set 
of all FSM states that can reach an error state. 
The system is safe if Y n f] 7 = 0. This check 
can be done in linear time. 

Several important issues concerning using MC to check 
safeness of control strategies are worth highlighting: 

• The large number of states in physical systems often 
forces one to use a reduced FSM model where some de- 
tails are abstracted out. Model checking cannot guar- 
antee safety under these circumstances. 



In our approach the EP algorithm renders FSMs which 
are complete in the sense that every aspect of the con- 
trol strategy is explicitly described in the FSM struc- 
ture. No details are abstracted out or reduced so the 
safety check results are guaranteed. 

• Model checkers typically provide trace information to 
help pinpoint where the safety property failed. 

We will not use this feature. In fact, we treat the en- 
tire safety issue as a decision problem — i.e., either the 
strategy is safe or it is not. Unsafe control strategies 
are immediately discarded, so there is no need to know 
why it is unsafe. 

• Model checkers are used to verify functional specifica- 
tions and other properties, e.g., liveness. 

In our approach model checking only verifies safety. 
All other performance criteria are assessed by trying 
out the control strategy in the physical environment. 

3. Implementation Details 

Our method can be summarized as follows: 

• Control strategies are encoded with FSMs. 

• An EP algorithm generates candidate control strategies 
by evolving FSMs. EP is ideally suited for this task [^] . 

• Safety properties are encoded as CTL expressions. 

• A symbolic model checker accepts the FSM and CTL 
expressions as input, and quickly checks to see if the 
control strategy is safe. The correctness of the safety 
check is guaranteed. 

• Safe control strategies are evaluated in the physical en- 
vironment whereas unsafe strategies are discarded. 

• The EP algorithm runs a fixed number of generations 
or terminates sooner if a suitable control strategy is 
found. The best performing FSM is implemented as 
the new control strategy. 
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